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Th«  Mc^imlcy  t»r  produt'iaie  iiorr  rtfllafciw.  laiMrr  cast  for 

•uch  Kltltary  applieatloa*  a»  cwiwi^nd  aad  cwierol.  aa^  avlaaira  tia«  lad  t« 
tha  daair*  to  davolop  oawar  leola,  ta<lMiii|vac,  aad  predtrtlaa  aids  for 
tuprovtna  cha  oatbud  to  tditrb  aoftwara  la  rarraocty  baloc  doaotapod  aod 
taatad.  This  daaira  baa  baan  aapraasad  la  sorb  docuavota  as  tba 
Jolac  LotlscUa  Cooaandara*  Softwara  Icltablllty  yortlap  Croup  loport 
(Rovaabar  1975)  aad  la  auaartMM  laduatry  aad  Covoroaoot  apoaaorod  roafaraocoa 
aad  ayapoaia.  Aa  a raauli.  affarta  baua  bacua  la  daaolop  aod  last  oatba- 
aactcal  aodala  for  pradlctlag  iba  arror  roataat  of  a aaftaara  parluaaa  aa 
wall  aa  datarolalat  last  critarla  aod  oaaawrva  of  caaplaslty.  itouaw»r, 
aarly  afforta  to  davalop  aucb  oodala  haaa  baae  frappaatad  aod  baaa  oat 
producad  aodala  witb  tba  daalrad  accuracy  aad  uaaabllllty. 

This  affert  waa  loltlatad  la  raapoaaa  to  this  aaad  for  daaatoptop 
accurate  aoftwara  predict Iva  aodala  aod  fits  lato  tba  goals  of  tAOC  TfO 
No.  5.  Software  Cost  Reductloe  (foraarly  RAMT  TfO  No.  11,  tofiaara  Scleocea 
Techaology),  la  particular  the  area  of  Software  Quality  (Software  Hodellog). 
This  report  auooarlaaa  tba  developoeat  of  oaaaures  of  language  aod  progroa 
cooplaalty  based  upon  the  appllcatloa  of  aatural  language  theory,  aod  In 
particular  ZlpCs  laws.  The  laportaace  of  this  deeel opaent  Is  that  It 
rapraaanta  tba  first  cohesive  at  leapt  to  apply  natural  language  theory  to 
the  davelopoant  of  software  cooplesliy  aeasurea,  which  In  turn  will  produce 
a cooprehenalve  theory  of  software  coopleslty  that  will  permit  prediction  of 
davalopmant  effort  and  error  content. 

The  theory  developed  under  this  effort  will  lead  to  much  needed  tools 
for  use  by  software  managers  la  adedwatefy  tracking  the  progress  of  a 
software  development  In  terms  of  the  sire  of  effort  or  nutber  of  errors. 

In  addition,  the  theory  developed  under  this  effort  will  provide  the  basts 
(or  further  development  Chat  will  eventually  produce  the  capability  to 
completely  define  and  measure  software  complexity.  Finally,  the  measures 
developed  under  this  effort  will  be  applicable  to  current  Air  Force  soft* 
ware  development  projects  and  thus  help  to  produce  the  high  quality,  low 
cost  software  needed  for  today’s  systems. 

a/lz.n.SM- 

ALAN  N.  SUKEAT 
Project  Engineer 
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ABSTRACT 


Th«rtf  i*  a sraal  oe«4  for  m«a»ur«»  of  protram  comploaiiy.  SocR 
meaaurvs  fiixf  appiUaiiao»  ioill  »arty  ^oaito  •aiimasa*  of  program  4iffiratty{ 
(2)  compariaoo  m«a*ur«»  of  program*;  ao4  (i|  a*  a oormaUaiot  paramo««r 
io  th«  analyat*  of  ««p«rim»oial  ^la  oo  error*  aod  ma«*hoor«.  W*  aSow 
that  using  «»tab)i*he4  runcepi*  of  »iail*lteal  languat*  iRoory  If^ipf  * tawf 
one  i*  able  to  4erive  formula*  for  program  com^etUly  similar  to  iAo*e 
obtained  by  tlal»t*ad^^  in  hi*  work  oo  aofiwore  ^y*ic*.  The  loehoito*  eaa 
be  uked  to  nteasure  and  study  language  compleaiiy  by  applying  il  lo  an  actual 
program  and  contparing  the  calculated  compleiUty  with  a theoretical  mini' 
mum  problem*  prog  ram  compleatly. 

W*  present  esperimeotal  dau  oo  PDP-11  accembly  language.  PLff 
program*.  MbSOO  assembly  language,  and  FORTRAN  •utemeot*.  udiich 
generally  verify  the  necessary  underlying  assumption  iZipTs  law  hold*  lor 
computer  languages).  Applying  ZIpT*  law  we  obtain  formula*  for  program 
leni^h  which  can  be  verified  by  comparison  wuh  statistics  obtained  from 
computer  program*. 

The  formulas  are  compared  with  Halstead's  worh  and  ours.  Both  set* 
of  length  formulas  agree  within  I0%-20%  of  the  actual  length  loperator* 
plus  operands)  although  neither  is  better  for  all  enample*. 


1.0  Introduction 

There  is  a great  need  for  theoretical  models  which  describe  program* 
and  allow  us  to  quantitatively  estimate  complenity.  running  time,  storage 
requirements,  and  development  lime.  In  addltloo  lo  serving  as  sn  estimate 
during  the  initial  design  period,  they  can  be  refined  ac  Die  program  develops 
and  used  as  a management  and  analysla  tool.  Tlicy  can  aleo  be  used  to 
compare  initial  design  approaclies.  programming  stylea*  different 
algorithms,  etc.  Early  work  oo  such  a Uicory  Is  given  la  Ref*.  19  and  20. 

This  report  is  closely  linked  to  Ref.  19  which  develop#  a theory  of 
program  complcnlty  based  on  linguistic  theory.  InformaDon  theory  aod  com- 
munications theory.  Tills  work  discusses  the  badtgrouod  linguistic  theory 
(Zlpfs  laws),  extends  t)iis  theory  to  programming  languages,  and  develop* 
equations  for  program  length  based  on  tli*  operatore,  operand*,  and  con- 
stants in  the  program.  T)\e  results  arc  different  from  tliosc  In  Halstead's 
work  on  Software  Physics*  *t  however,  in  many  example*  they  arc 
numerically  close.  The  main  differences  lie  In  the  basic  assumptions  and 
theoretical  development. 

2.0  Zipfs  Laws  of  Natural  Language 

2.  1 Introduction 


Any  investigator  who  has  studied  ways  of  describing  the  structure  of 
computer  languages  comes  to  the  immediate  conclusion  Uiat  t)»e  task  is  a 


I 


r 


r 
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formidable  one  due  to  the  seemingly  infinite  variety  of  programs  we  can 
write  with  a computer  language.  Much  ran  be  learned  in  tackling  this 
problem  by  studying  the  analogous  and  even  more  complex  problem  of 
describing  the  structure  of  a written  or  spoken  natural  language.  A set  of 
analogies  between  the  elements  of  natural  and  computer  languages  is  given 
in  Table  1. 

Table  1 « Analogies  between  elements  of  natural 
and  computer  languages. 


Natural  idterature  (or  dialog) 

Book  or  Speech 

Chapter.  Article,  Conversation 
Paragraph.  Section 
Sentence 

phrase,  clause 
subject 
predicate 
Word 
noun 

adjective,  adverb 
verb 

auxilary  verb 
preposition,  conjunction 
pronoun 
article 
Punctuation 
period 


Computer  Programs  (softwarel 

Software  System 

Program 

Subprogram,  Module.  Procedure 
Stateittent 
expression 

Element  belureen  spaces 
operand 
? 

ope raior 
? 

? 

? 

? 

Special  characters 

semicolon  iPLfll.CR.etc. 


Inspection  of  Table  1 shows  a parallelism  of  structure  between 
natural  and  computer  languages.  For  example,  nouns  correspond  to 
operands,  and  punctuation  marks  parallel  special  characters.  An  exact 
analogy  exists  if  we  describe  the  natural  language  using:  Chomsky 
grammar^Z  the  computer  language  using  BNF  form^^,  which  is 
essentially  an  adaptation  of  Chomsky's  work.  However,  it  will  only  be 
necessary  to  consider  words,  statements,  operators,  and  operands  in  the 
presentation  of  data  and  derivation  of  formulas  which  follows,  and  each  of 
these  is  easy  to  define  and  deal  with. 


The  reader  may  comment  that  natural  and  computer  laneuages  are  really 
different  things  and  like  apples  and  oranges  cannot  be  compared.  The  follow* 
ing  thought  problem  rebates  this  argument.  Suppose  we  take  a programmer 
who  understands  a particular  computer  language  and  give  him  a complete 
listing  with  comments  and  documentation  of  a computer  program.  We  instruct 


i 


him  to  study  the  computer  program  until  he  understands  it  and  then  produce 
a report  written  in  good  English  containing  paragraphs,  complete  sentences, 
and  algorithms  written  as  a sequence  of  steps  without  mathematical  notation. 
This  would  produce  a report  which  another  programmer  could  read  and  hand 
process  (perhaps  with  the  aid  of  a nonprogrammable  calculator)  to  produce 
an  output.  In  principle,  the  report  and  the  computer  program  would  be  equiv- 
alent. Thus,  any  studies  one  might  make  of  the  elements  of  the  computer 
program  would  have  to  bear  a close  resemblance  to  the  equivalent  elements 
of  the  report. 

2 Zipf's  First  Law 

Before  we  discuss  Zipf's  law  it  is  convenient  to  introduce  a few  terms 
used  in  dealing  with  natural  language.  Unfortunately,  the  term  word  is 
ambiguous  in  the  sense  that  all  the  words  in  the  abstract  of  this  report  mean 
something  different  than  all  the  words  in  the  dictionary.  We  use  the  term 
token  to  refer  to  all  the  words  of  the  abstract.  The  term  type  is  used  to  re- 
fer to  the  words  in  the  dictionary.  Much  effort  will  be  centered  on  the 

counting  of  n , the  number  of  times  the  r " particular  type  occurs  in  a 
sample  of  n 'tokens.  We  have  used  the  subscript  r because  in  our  compar- 
ison of  word  types  we  will  order  the  types  in  terms  of  their  frequency  and 
assign  ranks.  The  most  frequently  occurring  type  will  be  assigned  rank  1, 
the  second  most  frequent  type  rank  etc.  If  we  assume  there  are  t word 
types  in  our  sample  of  n tokens  then  clearly 

t 

y n = n (11 

f^l  *■ 

Clearly,  when  we  begin  to  collect  our  statistics  there  will  be  a number  of 
types,  t,  , which  occur  the  same  number  of  times,  k.  These  will  essen- 
tially tie  for  the  same  rank  r,  so  we  can  arbitrarily  assign  the  ranks  from 
r to  r-k+1  to  these  types  which  occur  k times.  Using  the  above  defini- 
tions we  can  write  a summation  equation  similar  to  Eq.  1 for  the  types 


V t =t  (2) 

k=l 


The  absolute  frequency  of  occurrence  for  type  r is  n^;  however  the  rela- 
tive frequency  of  occurrence  f^  is  simply  n^/n. 


Zipf  studied  the  relationship  between  frequency  of  occurrenc^  n^  and 
rank  r for  words  from  English,  Chinese,  and  the  Latin  of  Platus.  The 
resulting  data  is  given  in  Fig.  1,  where  Zipf  plotted  i.ne  log.Q  of  n^  on  the 
abcissa  and  the  log  _ of  r on  the  ordinate.  Note  that  the  data  for  English 
is  an  excellent  fit  ti^a  straight  line,  as  is  the  data  for  Latin  for  ranks 
la rge r than  1 0. 


1000 


RANK-r 


FIG.  1 OCCURRENCE  FREQUENCY  VS.  RANK  FOR  ENGLISH  AND 
LATIN  WORDS.  (FROM  ZIPF,  REF.  I,  PLATE  IV). 

If  we  had  plotted  logjQ  on  the  abcitaa  we  would  have  obtained  the  aame 
result  shifted  by  the  scale  factor  n.  The  straightness  of  such  plots  on  log* 
log  paper  is  an  empirical  justification  (we  will  discuss  later  theoretical 
justifications)  for  the  relationship 

n 

fj.  • r * Constant  * c . (3) 

Careful  study  of  Zipfs  data  and  that  of  others  shows  the  constant  a.  (the 
slope  on  log-log  paper)  is  approximately  1;  thus  we  arrive  at  the  simple 
relationship,  generally  called  ZipPs  Hrst  law,  given  below 

fj.  • r « c (4) 

which  can  also  be  written  in  the  form 


Initptfction  of  Eq.  <1  yield*  the  fact  that  the  cooatant  e can  he  inter' 
preted  a*  the  relative  frequency  of  the  rank  1 word  type.  There  are  come 
theoretical  and  practical  problems  with  this  obvious  Interpretation.  In  a 
practical  sense,  if  we  use  the  ratio  of  o}/n  to  determine  the  constant  c, 
we  will  probably  obtain  a poorer  estimate  of  c than  could  be  obtained  by 
measuring  the  y^axis  intercept  of  the  straight  line  fitted  to  the  dau  on  log' 
log  paper,  since  Zipf*  law  holds  less  well  at  times  for  the  lowest  rank. 
Theoretically,  we  can  examine  some  of  the  properties  of  the  frequency  dis* 
trlbution  to  see  if  we  can  interpret  it  as  a probability  distribution.  Letting 
Pr  be  the  probability  of  occurrence  for  the  rth  ranked  word,  the  summation 
of  all  the  Pr'*  yields 


(6) 


Substituting  Eq.  1 into  Eq.  6 we  obtain 


1 


(7) 


as  of  course  it  should. 


2.  1 Word  Type 


We  can  obtain  other  fundamental  relations  based  on.the  number  of  types 
via  summation  of  Zipfs  law.  If  we  sum  both  sides  of  Eq.  S we  get  a dif- 
ferent result. 


r»l 


n sen 
r 


I 

r 


(8) 


The  summation  of  the  series  1/r  is  given  by 


0.  5772  ♦ In  t 


♦Tt- 


I 

sees 

12t(tfl) 


Substitution  from  Eq.  1 and  Eq.  9 (retaining  only  2 terms  for  modest  size  t| 
into  Eq.  8,  and  rearrangement  yields  an  expression  for  the  constant  c in 
terms  of  t. 


1 

* 0.  5772  + In  t 


(10) 


L.  Jolley,  "Summation  of  Series,"  p.  36,  n.  200,  and  p.  14,  n.  70, 
Dover  Publications,  NY  1961.  (Note  the  constant  0.5772  is  called 
Eulers  constant,  see  "Differential  and  Integral  Calculus,"  R.  Courant, 
vol.  1,  Interscience  Publishers.  NY  1951,  p.  381,  420.) 
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l(  we  know  the  number  of  types,  t.  we  c»n  use  tq.  10  to  estimete  c.  iluw' 
ever,  we  notice  that  as  t increases,  c decreases.  From  £9.  4 f|  ei; 
thus,  the  frequency  of  the  first  rank  (and  all  the  other  ranks)  cttan^es  with 
c which  in  turn  varies  with  t.  However,  if  we  think  of  our  statistii  s as  a 
sample  and  apply  the  relative  frequency  interpretation  of  probability,  we 
would  expect  f to  approach  finite  values  as  the  santple  sire  (numt^r  of 
tokens,  n)  approached  infinity.  It  appears  that  in  natural  lanttuase  text, 
we  would  expect  t to  increase  with  n.  The  concept  of  P|  beinn  a function 
of  sample  size  is  contradictory  If  we  assume  random  statistically  indepen* 
dent  tokens  or  the  n tokens  represent  an  arbitrary  slice  from  a lariter 
work.  However,  if  we  require  that  the  n tokens  represent  some  cohesive 
unit  (e.  g..  abstract,  paragraph,  chapter,  paper,  etc.)  then  It  is  reasonable 
that  the  author's  style  and  work  probabilities  will  change  with  length.*  If 
such  is  the  case  we  would  need  data  for  different  token  lengths  and  their 
associated  values  of  c so  that  we  would  investigate  the  validity  of  Eq.  10. 
The  way  in  which  t varies  with  sample  size  n is  stated  by  examining  how 
the  type-token  ratio,  t e t/n.  varies  with  n. 

We  can  derive  an  equation  for  the  type-token  ratio  by  considering  the 
behavior  of  Eq.  5 for  the  smallest  rank  .which  is  where  r » t (e.  g.  . if 
there  are  100  types,  then  the  largest  rank  is  obviously  100).  In  most  cases 
the  rarest  type  (largest  rank)  will  occur  only  once;  thus.  ntmMx  * 
stituting  these  values  in  Eq.  5 yields  another  interpretation  Tor  the  con- 
stant c 


tsI  c 
n 


Equating  Eq.  10  to  Eq.  1 1 yields 


0. 5772  + In  t 


and  we  now  have  an  expression  for  the  dependence  of  t on  t.  In  order  to 
relate  t to  n we  substitute  t = nr  in  Eq.  12  and  solve  for  n obtaining 


, ( - . 0.  57721 

1 T 
n = -e 


Clearly,  n increases  with  decreasing  t or  equivalently  t decreases 
with  n.  The  shape  of  this  relationship  between  T and  n agrees  well  with 
the  shape  of  the  sparse  data  in  the  literature;  however,  the  numerical 
values  are  in  poor  agreement.**  We  can  obtain  an  equation  relating  n to 
t by  substituting  t/n  for  T in  Eq.  1^  and  solving 

n = t(0.5772  + Int)  . (14) 


ef.  Ref.  15  p.  245. 

See  Miller  Ref.  4,  p.  124. 
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2.4  Zipr»  Second  Law 

We  can  derive  a aecund  form  of  7.ipV»  law  from  the  first  form  by 
making  other  assumptions  about  the  behavior  in  the  region  where  r is 
larger.  * We  compute  the  derivative  of  with  respect  to  r fron 

Eq.  5 and  take  its  magnitude 


(15) 


Eliminating  r by  substitution  from  Eq.  5 into  Eq.  1 5 yields 

dn 

r 
dr 


In  the  tail  of  Zipfs  law,  there  are  several  identical  n values;  thus  there 
is  a plateau  of  k types  with  the  same  n value  which''are  tied  for  the  same 
rank.  The  curve  becomes  a staircase  function  and  we  define  the  slope  as 
the  vertical  decrement  (r  to  r 1 £ 1)  over  the  horizonul  width  (which  is 
t|^),  thus 


r 

cn 


(16) 


dn 
r 

dk 


(17) 


In  the  tail  f'rmax"  ^ h = 1,  ''r/^^x-l)  “ ^ k = 2,  thus  in  this  region 
n = k,  and  suDstituting  into  Eqs.lv  'and  16  yields  the  second  form  of 
Zipf  s law. 


(18) 


Experimental  verification  of  Zipfs  second  law  is  given  in  Figs.  2,  3, 
and  4 . We  can  also  derive  a second  set  of  basic  equations  using  the 
second  law.  Combining  Eq.  2 with  18  we  obtain 


t = 


(19)* 


"Handbook  of  Mathematical  Functions,"  NBS,  1964,  p.  807,  see 
Riemann  Zeta  function. 


7 


NUMBER  OF  WORDS 


FIG.  2.  EXPERIMENTAL  VERIFICATION  OF  ZIPF'S  SECOND  LAW, 
(Chinese  Words) 


NUMBER  OF  WORDS 


FIG.  3.  EXPERIMENTAL  VERIFICATION  OF  ZIPF'S  SECOND  LAW. 
(English  Words) 


lOOl 


! 


FIG.  4.  EXPERIMENTAL  VERIFICATION  OF  ZIPF'S  SECOND  LAW. 
(Latin  Words) 


Combining  Eq.  19  and  10  yields 

n = (0.  5772  -f  1 n t)  t (20) 

w 

and  by  substituting  t s nr  in  Eq.  20  and  solving  we  obtain 

1 

ns—®  • 

T 

Note  that  except  for  a factor  of  ir  /6  the  results  given  ^ Eqs.  13  and  21  and 
14  and  20  are  identical.  The  occurrence  of  either  6/ir^*  0.  6079  or  unity 
as  a coefficient  in  these  equations  arises  due  to  the  slightly  different 
assumptions  made  concerning  the  behavior  of  Ziprs  law  for  small  ranks 
in  the  two  derivations. 

Since  Zipfs  second  law  is  derived  from  the  first,  we  can  verify  the 
fact  that  a is  unity  ( cf.  Eq.  3)  by  plotting  f^.  vs.  f(  cf.  Eq.  4)  on  log -log 
paper  or  tj^  vs.  ( cf.  Eq.  19)  and  checking  for  a unity  slope.  The 
z-axis  intercept  is  the  value  of  the  constant  c in  the  first  law  plot,  and  the 
intercept  of  the  second  law  plot  is  cn.  From  Eq.  17  we  see  that  the 
1 ^ second  law  contains  the  derivative  of  the  first  law;  thus,  the  statistical  fluc- 

l tuations  of  the  data  will  be  accentuated  in  the  second  law  plot.  (Thus  the  first 

• law  is  better  for  curve  fitting.  ) 


(21) 


Zlpft  law  (or  Zipf  type  laws)  have  shown  to  apply  to  a wide  variety  of 
different  phenomenal'*  ^ * 


(1)  The  population  of  the  larger  cities  (even  better  for  metropolitan 
districts)  for  every  U.  S.  census  since  1790,  also  all  cities  of 
Europe  (but  not  for  Great  Britain). 

(2)  If  the  constant  a is  equal  to  l/2,  the  law  holds  for  the  income  of 
people  in  the  U.  S.  (called  Pareto's  law). 

(3)  The  product  of  the  number  of  students  attending  a university 
from  a particular  state  and  the  distance  of  the  state  from  the 
university  is  proportional  to  the  population  of  the  state  (expert* 
mentally  verified  for  Harvard,  MIT,  and  Princeton). 

(4)  The  same  law  (as  3)  applies  to  number  of  charge  accounts  at 
Jordan  Marsh  Co.  in  Boston  in  various  cities  and  towns  of  New 
England. 

(5)  The  "interchange"  between  the  cities  is  proportional  to  the 
product  of  the  populations  divided  by  the  distance  between  them 
and  applies  to  Telephone  Calls.  Railway  Express  packages. 
Truck  and  Passenger  trips,  and  many  other  things. 

(6)  Many  different  languages  and  language  elements. 

Thus,  Zipfs  law  seems  to  hold  for  a wide  variety  of  "organised"  behavior, 
of  both  humans  and  machines. 


Lk 


2.5  Generalized  ZipPs  Law 

The  fact  that  Zipfs  law  in  its  simple  form  fits  many  sets  of  data  leads 
to  the  rather  sirrple  and  powerful  theory  developed  in  Sections  2.  2-2.4.  In 
many  cases,  the  experimental  fit  to  Zipfs  first  law  is  only  close  enough  for 
the  ensuing  theory  to  be  thought  of  as  a gross  approximation.  In  such  cases, 
two  generalized  forms  of  Zipfs  laws  will  yield  a better  fit  between  theory 
and  data.  In  such  cases,  the  algebraic  expressions  and  computations  be- 
come more  cumbersome.  However,  a simple  computer  program  solves 
the  computational  problem  and  parametric  studies  provide  sensitivity  in- 
formation on  how  the  results  change  as  parameters  vary.  Of  course  the 
simple  Zipf  theory  will  still  hold  well  enough  to  provide  some  insight  into 
the  results. 

One  generalized  form  of  Zipfs  law  (suggested  by  several  authors)  is  of 
the  form 

n ^ 


Pr=  TT  = 


(r  + A) 


B 


(22) 


We  can  easily  see  the  role  which  the  constants  play  by  taking  logarithms  of 
both  sides  of  Eq.  22  and  rearranging  them. 


10 


In  ■ In  C-B  In  (r*A) 


liW 


clearly  if  Aa  0 and  B ■ 1 w«  obtain  Zipfa  first  law.  The  constant  B 
represents  the  slope  of  the  plot  and  A is  a "shift*  paranneter.  For  con* 
venience  we  will  call  this  the  slope -rank  shift  form.  The  role  of  A can  be 
illustrated  if  we  set  B ■ 1 and  r a 1 

In  p^  a In  C • In  (l^A)  a in  ( ) (^4| 


If  AaO.  then  C is  the  frequency  of  the  first  rank.  i.  e.  . the  y axis  intercept. 
If  AaO.  1 the  intercept  is  about  1.  1C  and  if  Aa«0.  1 the  intercept  is  about 
0.  9C.  Examination  of  Eq.  23  shows  that  for  large  r.  the  effect  of  A is 
negligible.  These  results  are  illustrated  In  Fig.  5.  where  we  see  that  B 
allows  us  to  match  a general  slope  other  than  unity  and  A adjusts  for 


FIG.  5.  ROLE  OF  COEFFICIENTS  A,  ^C  IN  EQ.  23. 

derivations  from  straight  line  behavior  for  the  initial  ranks. 

The  piprsical  meaning  of  the  constants  A.B.C  have  been  studied  in 
linguistics^  and  the  following  observations  have  been  noted: 

(1)  Values  of  B greater  than  unity  occur  in  individuals  who  posses 
a limited  vocabulary,  such  as  young  children. 

(2)  Values  of  B less  than  unity  are  associated  with  large  vocab* 
ularies,  e.  g.  , James  Joyce  or  collections  of  writing  by  various 
authors  as  in  a newspaper  or  anthology. 


11 


ill. 


1 


<3)  Value*  of  A greater  than  aero  occur  tf  the  prubabUtty  of  the 
first  rank  word  is  sntall  as  is  the  case  In  English  of  words  fol' 
lowing  "the",  or  in  a highly  inflected  language. 

(4)  Value*  of  A less  than  aero  occur  If  the  probability  of  the  first 

word  is  large,  as  in  the  case  of  words  following  "of*  (*the*  being 
very  frequent  here). 

In  order  to  derive  the  modified  basic  equation*  for  the  slope>rank  shift 
form  of  the  law,  we  will  apply  Eq.  1 to  Eq.  IZ 


I «, -Cn  Z 


I 


rel 


rel  (r+A) 


F 


(25) 


we  can  sum  the  above  series  with  the  aid  of  the  Euler- Maclaurin  Formula 
which  relates  finite  sum  of  a function  to  the  integral  of  the  function. 


n s 


12 

B 

.B 

12A^+6(B43)A  ♦B^45B46  1 

B^+7B+12 

X 

12(B-1)(A+1)®'*^*  (B-l)(t+A)®^V 

(26) 


For  the  special  case  where  B 1 


n 


ln(t+A)  - ln(A  + l)4 


6A47 

I2(A+1)^. 


and  where  A s 0 


12 

B 

.B 

B^+5B+6  1 

B^+7B+12 

t 

[ '2(B.1)  (B-Dt®-' 

Similarly  one  can  show  that 


cn 


12t 

B^+7B  + 12 


B 


* 


(27) 


(28) 


(29) 


See  Ref.  14,  p.  322 


12 


(B^  *78  + 12) 


(J21 


When  we  plot  Zlpf  law  data,  aometimea  we  have  found  experiment*!  reaulta 
on  log'log  paper  which  look  like  the  atraight  llnea  with  different  alopea. 
(The  reader  who  ia  familiar  with  the  control  ayatem  literature  will  notice 
the  analogy  with  Bode  plota.  ) Limiting  our  model  to  two  piecewiae 'linear 
aegmenta  (aee  Fig.  6)  we  obtain 


'1 


B. 


1 < r <D 


(r) 

C, 


(33) 


B. 


D < r < t 


(r) 


(D) 


(D) 


Again,  by  uaing  the  low  order  terms  in  the  Euler-Maclaurin  formula,  on 
the  two  segments 


B. 


n = 


12t 


iB^+VB^+lZ 


Bj+5B^+6 


B^-Bi 


12(Bj-l)D 


B^-Bi 


B,.l 


(Bj-IXB^-DD  ^ (B2-l)t 


B,-l 


(34) 
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papc-'r  fitii  /ipi'tf  lawt>  and  ty  prtxJtt litm  accuracy  uf  any  (urmulaa  dn* 
vvlupvd  in  th«  analyaia. 

i.  d Kxih?  rimantat  Evidcoca 

We  have  examined  a number  ot  computer  protrama  and  al»ay«  find  that 
some  form  ofZipfb  law  eeeme  to  fit  the  data  fairly  well.  The  data  are 
listed  in  Tables  2 to  li  and  the  7-ipf  plots  are  given  in  Figs.  7 to  17. 

Table  d • Experimental  Data  on  Computer  Programs 


and 

ZipPs  Laws 

Program 

Figures 

Types 

Tokens 

1. 

PL/I  Fibonacci: 

7 

Operators 

12 

31  Operators 

Numbers  Program  • 11 

8 

Operands 

9 

24  Operands 

Lines  long,  on  p.  81 

9 

Operators  b 

21 

55  Total 

of  Ref.  8 

Operands 

2. 

PL/1  Student  Grades 

10 

20 

1 17  Operators 

Program  - 27  lines  long. 

11 

31 

105  Operands 

on  p.  178  of  Ref.  8 

12 

51 

222  Toul 

L 

MIKBUG  - Executive 

1 3 

25 

190  Operators 

Program  for  Mb  micro- 

62 

1 32  Operands 

processor 

87 

322  ToUl 

4. 

PDP-11  Assembly  Language 
Programs 

14 

Operators 

39 

1572  Tokens 

5. 

Variable  Names  in  3 PL/I 
Programs 

15 

Codes 

50,  24. 

34  370,238.193 

6. 

FORTRAN  Statement  Types 

16 

40 

200k.  and  10k 

(Table  2,  Ref.  18)  17 


In  program  No.  1 we  have  separately  analyzed  operators,  operands,  and 
combined  operators  and  operands,  j^e  Tables3.  4.  S and  Figs.  7.8.9.) 

Note  that  we  have  followed  Halstead!'  in  our  definition  of  operators 
and  operands.  Basically  in  PL/1  operators  Include:  key  words,  comparison 
operators.  and  The  operands  include:  variables  and  constants. 

Excluded  from  both  lists  are  declare  statement  and  comments.  Labels  are 
considered  part  of  the  GO  TO  statement,  i.  e.  , GO  TO  Label  1 is  an 
operator.  Labels  which  are  not  used  are  ignored  as  are  comments.  Note 
that  a simple  Zipf  s law  fits  operators  and  operands  well  (cf.  Figs.  7,8); 
however  the  combination  of  operators  and  operands  is  better  fitted  by  the 
first  generalized  form  of  Zipfs  law  ( cf.  Fig.  9). 

In  program  No.  2 we  find  that  neither  the  operators  ( cf.  Table  6)  nor 
the  operands  ( cf.  Table  7)  fit  ZipTs  law  too  well  ( cf.  Figs.  .10,  11). 

15 


"1 


i 


Table  3 • Rank'Prubability  Oals  for  Operalure  of 
Program  No.  I in  Table  t 


r«Rank 

D^'ProbabilUy 

Symbol 

Count 

1 

0.  35 

e 

• 

11 

2 

0.  16 

■ 

5 

3 

o 

* 

e 

0 

3 

4 

0.065 

1 

2 

5 

0.065 

PUT  LIST 

2 

6 

0.065 

♦ 

2 

7 

0.032 

PROCEDURE 

1 

8 

0.032 

GET  LIST 

1 

9 

0.0  32 

IF  THEN 

1 

10 

0.032 

< 

1 

ll 

0.032 

CO  TO 

1 

12 

0.032 

END 

1 

ToUl 

31 

Table  4 - Rank -Probability  Data  for  Operand#  of 
Program  No.  1 In  Table  2 

r-Rank 

D -Probability 

Symbol 

Count 

1 

0.25 

LAST-F 

6 

2 

0.  17 

N 

4 

3 

0.  17 

PREV  F 

4 

4 

0.083 

FIBONACCI 

2 

5 

0.083 

LIMIT 

2 

6 

0.083 

REPEAT 

2 

7 

0.083 

TEMP 

2 

8 

0.  042 

2 

1 

9 

0.042 

1 

1 

ToUl  24 


16 


0 


1 


a 


di 


F 


f».4  language  Theory  Meagureg  of  Complexity 

The  use  of  nunber  of  statements  as  a measure  of  complexity  Is  often 
used.  It  Is  recognized  that  such  a simplistic  measure  does  not  truly  gauge 
the  program.  Often  there  are  programs  with  a few  lines  considerably  more 
complex  than  programs  with  many  lines.  As  a better  measure  Halstead  (Refer- 
ence 27)  suggested  operator  and  operand  count.  The  analogy  between  operators, 
operands  and  verb  nouns  suggested  the  application  of  Zlpf's  Law  from  natural 
languages  to  programming  languages. 

The  relationships  between  nunber  of  operators  and  operand  types,  as 
well  as  their  frequency  of  occurrence,  was  shown  to  follow  the  basic  Zlpf's 
Law.  An  extended  form  of  Zlpf's  Law  has  been  derived  which  more  closely  j 

models  some  of  the  experimental  data. 

The  theory  culsilnates  in  an  equation  which  relates  operator  and  operand 
length  to  the  nunber  of  types  (References  9 and  25).  If  we  estimate  early  In 
the  design  the  nunber  of  Input  and  output  varlbles  and  operators  (which  will  ' 

be  used  in  the  program)  we  can  obtain  an  estimate  of  the  program  length.  i 

These  results  correlate  well  with  the  results  which  Halstead  has  obtained 
using  software  science  techniques  (Reference  21,  Chapter  3,  and  Reference  27).  ' 


Table  6 • Rank -Probability  Data  for  Operators 
of  Program  No.  i in  Table  i 


r-Rank 

0 -Probabilitv 

Sv  mbol 

Count 

1 

0.  31 

m 

36 

Z 

0.  23 

• 

28 

3 

0.  14 

IF  THEN 

17 

4 

0.04  3 

f 

5 

5 

0.0  34 

SUBSTR 

4 

6 

0.026 

• 

• 

3 

7 

0.  026 

( ) 

3 

8 

0.  026 

SKIP 

3 

9 

0.026 

+ 

3 

10 

0.  026 

PUT  LIST 

3 

11 

0.017 

1 1 

2 

12 

0.017 

GO  TO 

2 

13 

0.  0085 

PROCEDURE 
OPTIONS  MAIN 

1 

14 

0. 0085 

PAGE 

1 

15 

0. 0085 

LINE 

1 

16 

0. 0085 

GET  LIST 

1 

17 

0.  0085 

=!■ 

1 

18 

0.  0085 

/ 

1 

19 

0.  0085 

- 

1 

20 

0.0085 

END 

1 

Total 

117 

r 


Table  7 • Rank-Probability  Data  for  Operands 
of  Program  No.  2 in  Table  2 


• Rank 

p •Frooabilltv 

Symbul 

Cou«£ 

1 

o..otfe 

1 

9 

0.057 

FLNAL- LETTER- 
GRADE 

6 

) 

0.057 

STUDENT  DATA 

8 

4 

0.057 

MtD.TER.M 

0 

» 

0.057 

Fl.N’AL 

8 

0 

0.057 

.MIDTER.M-TEST 

O 

7 

0.057 

FINAL- TEST 

b 

it 

0.057 

FINAL-GRADE 

0 

<> 

0.018 

POSITION 

4 

10 

0.018 

) 

4 

11 

0.018 

2 

4 

12 

0.018 

0 

4 

11 

0.02'> 

B 

1 

14 

0.021 

A 

1 

15 

0.029 

4 

1 

le> 

0.029 

B 

3 

1“ 

0.029 

C 

1 

18 

0,029 

D 

1 

1» 

0.029 

F 

1 

20 

0.019 

FINAL-GRADES 

2 

21 

0.019 

> 

22 

0.019 

STUDENT-NA.ME 

2 

21 

0.  019 

PROCESS-.NE.NT- 

STUDENT 

> 

24 

0.019 

TERMLNATE 

2 

25 

0.009  5 

•THE  FINAL.  ..  COURSE'  1 

26 

O 

o 

o 

10 

1 

27 

0.  00>' 

8 

I 

28 

0.0095 

44 

1 

2? 

0.0015 

- 

1 

30 

0.  0095 

0.  5 

1 

31 

0.  0095 

INDEX 

Total 

_1_ 

105 

Table  8 - Rank-Probability  Data  for  Operators  Plus 
Operands  of  Program  No.  2 in  Table  2 


r-  Rank 

D-  Probabllilv 

Symbol 

Count 

1 

0.  16 

s 

36 

2 

0.  I 3 

28 

3 

0.077 

IF  THE.N 

17 

4 

0.041 

1 

9 

5 

0.027 

FINAL- LETTER-GRADE 

6 

6 

0.  027 

STUDENT-DATE 

6 

7 

0.027 

MID-TERM 

6 

8 

0.027 

FINAL 

6 

4 

0.027 

.MIDTERM-TEST 

6 

10 

0.027 

FLN’AL-TEST 

6 

11 

0.027 

FUNAL-CRADE 

0 

12 

0.  023 

5 

1 3 

0.018 

SUBSTR 

4 

14 

0.  018 

POSITION 

4 

15 

0.018 

3 

4 

16 

0.018 

2 

4 

17 

0.018 

0 

4 

18 

0.014 

3 

19 

0.014 

( 1 

3 

20 

0.014 

SKIP 

3 

21 

0.014 

+ 

3 

22 

0.014 

PUT  LIST 

3 

23 

0.014 

b 

3 

24 

0.014 

A 

3 

25 

0.014 

4 

3 

26 

0.014 

B 

3 

27 

0.014 

C 

3 

28 

0.014 

D 

3 

29 

0.014 

F 

3 

30 

0.  009 

1 1 

2 

31 

0.  009 

CO  TO 

2 

32 

0.  009 

FINAL-GRADES 

2 

33 

0.009 

1 

2 

34 

0.  009 

STUDENT-NA.ME 

2 

35 

0.009 

PROCESS-NEXT-STRLNG 

2 

36 

0.009 

TER.MINATE 

2 

37 

0.  0045 

PROCEDURE  OPTIONS  MALN 

1 

38 

0.  0045 

PAGE 

1 

39 

0.  0045 

LINE 

1 

40 

0. 0045 

GET  LIST 

1 

41 

0.  0045 

# 

1 

42 

0.  0045 

/ 

I 

43 

0. 0045 

- 

1 

44 

0.  0045 

END 

1 

45 

0.  0045 

' THE  FINAL.  . . GRADE' 

46 

0.0045 

10 

1 

47 

0.  0045 

8 

1 

48 

0. 0045 

44 

1 

49 

0.  0045 

. 

1 

50 

0. 0045 

0.  5 

1 

51 

0.0045 

INDEX 

1 

Total  221 
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T^ble  9 - Liisting  of  Fibonacci  Numbers  PL/I  Program 
(p.  81  of  Ref.  8) 


FIBONACCI; 


REPEAT; 


PROCEDURE; 

N = 2; 

GET  LIST  (PREV_F,  LAST_F,  LIMIT); 
PUT  LIST  (PREV_F.  LAST_F); 

TEMP  = LAST_F; 

LAST_F  = LAST_F  + PREV_F; 

PREV_F  = TEMP; 

PUT  LIST  (LAST_F); 

N = N+1; 

IF  N < LIMIT  THEN  GO  TO  REPEAT; 
END  FIBONACCI; 
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FINAL-GRADES:  »ROCEOcRE  C’ltOsS  (/.'Alt,' 

/•  THE  t^CXEDfRE  CALCULATES  THE  FIT.al  GRADE'  THAT  v.IlL  BE  GIVEN  IN  , 
Pl/1  COJRSF.  USING  THE  Lt’IER  GFACiS  TE^EiVED  IN  I“E  /.‘IDTERV  TEST 
AND  THE  final  TEST  O'  THE  COURSE,  THE  FINAL  ORAU  IS  CALCULATED 
FOR  EACH  student. 

/•  THE  NAA>E  AND  THE  GRADES  O'  A STUDENT  ARE  TEH  IN  A CH.VRAC^ER 
STRING  variable.  " ^ 

DECLARE  student-data  CHAR  (30)  ''ARY INC; 

DECLARE  POSITION  FIXED  DECIMAL;?, 0);  /*  INDEX  INTO  THE  DATA.  V 

/•  THE  LETTER  GRADES  ARE  PLACED  IN  CHARACTER  STRING  /A’lAoLES  OP 
LENGTH  ONE.  NUMERIC  GFADES  ARE  » EPT  IN  INTEGER  VARIASLES.  '/ 
DECLARE  (MIDTERM,  FINAL,  FKJAL-LETTER-GRADE)  CHAR  'll, 

DECLARE  (MIDTERM-TEST,  FINAL-TEST,  FINAL-GRADEI  FiXEC  DEC  (2); 

A character  string  variable  O'  FIXED  LENGTH  IS  USED  FOR  THE 
PRINTING  OF  THE  STUDENT  NAMF.  IT  V.ILL  FRC/IDE  ALIGNMENT  IN  THE 
OUTPUT  STREAM.  • ' 

DECLARE  student-name  CHARACTER  (15); 

/*  PREPARE  THE  PAGE  HEADING.  */ 

PUT  LIST  ('THE  FINAL  GRADES  RECEIVED  IN  THE  Pl/I  CO«JRSE')PAGE  LINE(IO), 
PUT  LIST  f19)'  ' 11  (44)'  - •)  SKIP(O); 

PUT  SKIPO); 

PROCESS-NEXT-STUDENT: 

/•  ACQUIRE  The  DATA  OP  ONE  STUDENT.  */ 

GET  LIST  (STUDENT-DATA); 

/*  THE  END  OF  THE  INP'JT  STREAM  IS  REACHED  V/HEN  A NULL  STRING  IS 
ENCOUNTERED  DURING  THE  DATA  ACQUIS ITIO.N  '/ 

IF  STUDENT-DATA  = " THEN  GO  TO  TERMINATE; 

/•  LOCATE  THE  Ei-u  O^  THE  STUDENT  NAME.  V 
POSITION  = INDEX  (STUDENT-DATA, ', '); 

/•  LOCATE  THE  MIDTERM  AND  THE  FINAL  TEST  RESULTS.  •/ 

MIDTERM  = SUBSTR  (STUDENT-DATA,  POSITION  i 1,  i); 

FINAL  = SUBSTR  (STUDENT-DATA,  POSITION  t 3,  I); 

/*  CONVERT  THE  MIDTERM  AND  THE  FINAL  TEST  RESULTS  INTC  NUMERIC 
GRADES.  V 

IF  MIDTERM  = 'A'  THEN  MIDTERM-TEST  = 4- 
IF  MIDTERM  = 'B'  THEN  MIDTERM-TEST  ^ 3; 

IF  MIDTERM  = 'C  THEN  MIDTERM-TEST  r 2; 

IF  MIDTERM  = 'D'  THEN  MIDTERM-TEST  ^ 1; 

IF  MIDTERM  = 'F'  THEN  MIDTERM-TEST  = 0; 

IF  FINAL  .=  'A'  THEN  FINAL-TEST  - 4: 

IF  FINAL  = 'B'  THEN  FINAL-TEST  = 3, 

IF  FINAL  = 'C  THEN  FINAL-TEST  = 2, 

IF  FINAL  = 'D'  THEN  FINAL-TEST  = I 
IF  FINAL  = 'F'  THEN  FINAL-TEST  = 0, 

/•  CALCULATE  THE  FINAL  GRADE  THE  STUDENT  WILL  RECEIVE.  V 
FINAL-GRADE  = (MIDTERM-TEST  +2  * FINAL-TEST) /3  +0.5; 

/•  DETERMINE  THE  FINAL  LETTER  GRADE.  */ 

IF  final-grade  = 4 THEN  F INAL-LEHER-GRADE  = 'A'; 

IF  FINAL-GRADE  = 3 THEN  FINAL-LEITER-GRADE  = 'B'; 

IF  FINAL-GRADE  = 2 THEN  FINAL-LETTER-GRAOE  - 'C, 

IF  FINAL-GRADF.  = 1 THEN  FINAL-LETTER-GRADE  - 'D'; 

IF  FINAL-GRADE  = 0 THEN  FINAL-LETTER-GRADE  . 'F'; 

/•  DISPLAY  THE  FINAL  GRADE  OF  THE  STUDENT.  */ 

STUDENT-NAME  = SUBSTR  (STUDENT-DATA,  I,  POSITION -1); 

PUT  LIST  (",  STUDENT-NAME  II  FINAL-LETTER-GRADE)  SKIP; 

GO  TO  PROCESS-NEXT-STUDENT;  /*  GET  THE  DATA  Of  THE  NEXT  STUDENT.  */ 

TERMINATE:  END  FINAL-GRADES; 


Table  10  - Listing  of  Students  Grades  PL/I  Program 
(p.  178  of  Ref.  8) 


* 


FIG.  8.  ZIPF  LAW  PLOT  FOR  OPERANDS  IN  PROGRAM  NO.  1 OF  TABLE  2. 
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. ZIPF  LAW  PLOT  FOR  COMBINED  OPERATORS  AND  OPERANDS  IN 
PROGRAM  NO.  1 OF  TABLE  2. 
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FIG.  10.  ZIPF  LAW  PLOT  FOR  OPERATORS  IN  PROGRAM  NO.  2 OF  TABLE  2 


FIG.  13.  RELATION  BETWEEN  THE  RANK  AND  COUNTS  OF  OPERATORS 
OPERANDS,  AND  THEIR  TOTAL. 


TABLE  2 (LOCKHEED  DATA). 
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TABLE  2 (STANFORD  DATA). 


Table  12  - Rank- Frequency  Data  for  FORTI^N  Statement 
Types  for  Program  5 of  Table  2 


■Lockheed  Data 


Stanford  Data- 


Statement  Type 

Rank 

Probability 

Number 

Rank 

Probability* 

Numt 

As  signment 

1 

0.  38 

78435 

1 

0.  50 

4869 

IF 

2 

0.  14 

27967 

2 

0.  084 

816 

GO  TO 

3 

0.  120 

24942 

3 

0.  080 

777 

Call 

4 

0.  073 

15125 

7 

0.  035 

339 

Continue 

5 

0.  038 

9165 

8 

0.  032 

309 

Write 

6 

0.  0 37 

7795 

4 

0.  052 

508 

Format 

7 

0.  037 

7685 

6 

0.  039 

380 

DO 

8 

0.  6 36 

7476 

5 

0.  047 

457 

Data 

9 

0.  021 

4468 

18 

0.  0029 

28 

Return 

10 

0.  018 

36  39 

10 

0.  019 

186 

Dimension 

1 1 

0.  017 

3492 

11 

0.  015 

141 

Common 

12 

0.  014 

2908 

9 

0.  027 

26  3 

End 

1 3 

0.  012 

256  5 

12 

0.  012 

121 

Buffe  r 

14 

0.  012 

2501 

- 

- 

0 

Subroutine 

15 

0.010 

2001 

15 

0.  0095 

93 

Rewind 

16 

0.  0083 

1724 

23 

0.  00062 

6 

Equivalence 

17 

0. 0066 

1382 

13 

0. 0012 

113 

Endfile 

18 

0.  0037 

76  5 

29 

0.  0002 

2 

Integer 

19 

0. 0032 

6 57 

17 

0.  0035 

34 

Read 

20 

0. 0028 

586 

16 

0.  0094 

92 

Encode 

21 

0.  0028 

58  3 

- 

- 

0 

Decode 

22 

0. 0027 

557 

- 

- 

0 

Print 

23 

0. 0017 

345 

25 

0.  00052 

5 

Entry 

24 

0. 0013 

279 

20 

0. 0015 

15 

Stop 

25 

0. 0091 

190 

21 

0. 0011 

11 

Logical 

26 

0. 0082 

170 

22 

0.  0009 

9 

Real 

27 

0. 00071 

147 

28 

0.  0003 

3 

Ident 

28 

0. 00051 

106 

- 

- 

0 

Overlay 

29 

0. 00039 

82 

- 

- 

0 

Pause 

30 

0. 00027 

57 

24 

0.  0006 

6 

As  sign 

31 

0.  00027 

57 

27 

0.  0004 

4 

Punch 

32 

0. 00025 

52 

26 

0.  0005 

5 

Exte  rnal 

33 

0.  00011 

23 

31 

0. 0001 

1 

Complex 

34 

0. 000029 

6 

- 

- 

0 

Name  List 

35 

0. 000024 

5 

- 

- 

0 

Double 

36 

0. 000014 

3 

14 

0.  010 

99 

Block  Data 

37 

0. 0000048 

1 

30 

0.  0002 

2 

Implicit 

- 

- 

0 

19 

0. 0016 

16 

Input 

- 

- 

0 

- 

- 

0 

Output 

- 

- 

0 

- 

- 

0 

TOTAL 

207.941 

9710 

Comment 

(28) 

52,  924 

(11) 

1090 

Continuation 

(7) 

(7) 
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Knuth  counted  IF  statements  twice,  thus  the  total  is  actually  higher  than  the 
number  of  statements. 


However,  a reasonable  fit  is  obtained  for  their  sum  ( cf.  Table  8 and  Fiji. 
12).  The  listings  of  programs  No.  1 and  2 of  Table  2 appear  in  Tables  H 
and  10. 

The  MIKBUG  is  a TTY  handler  and  simple  debugger  for  the  M6800 
microprocessor.  Fig.  13  shows  the  relations  between  the  rank  and  the 
counts  of  operators  (OPcodes),  operands  (variable  names  and  statement 
labels)  and  their  total.  Fig.  14  shows  the  relation  between  the  rank  and  the 
OP  code  counts  for  several  PDP-11  programs.  Both  of  these  examples  are 
in  assembly  language.  As  in  most  higher  languages,  the  upper  part  of  the 
operand  plot  is  flattened,  and  the  lower  part  of  the  operator  plot  drops  off 
more  rapidly  than  the  upper  part. 

Three  PL,/I  programs  are  plotted  in  Fig.  15.  Only  variable  names  are 
shown.  The  flattened  upper  part  can  again  be  seen.  In  one  respect  com- 
puter operands  differ  quite  markedly  from  their  natural  language  analog 
nouns,  or  from  words  in  general.  The  second  form,  or  low  frequency  form, 
of  Zipf's  law  is  not  followed  that  well  in  the  computer  programs. 

We  have  explored  how  well  PL/l  and  assembler  operators  and  operands 
are  fitted  by  Zipf's  laws  and  their  extensions.  One  can  also  explore  how 
well  statement  types  in  FORTRAN  fit  Zipf  s law.  Knuth  ( cf.  Ref.  18) 
studies  data  on  about  200,000  FORTRAN  statements  collected  at  Lockheed 
Aircraft  and  10,000  at  Stanford.  His  data  is  converted  to  Rank-frequency 
data  in  Table  12  and  is  plotted  in  Fig.  l6  and  17.  Note  that  in  Table  12  the 
statement  types  rank  differently  for  the  Lockheed  and  Stanford  data. 

(1)  The  top  10  ranks  are  quite  similar  except  for  "DATA". 

(2)  Similarly  the  top  20  are  the  same  except  for  three  exceptions 
(14,  23.  29). 

(3)  Stanford  Programmer s are  either  unaware  or  choose  not  to  use 
ENCODE.  DECODE,  IDENT,  and  OVERLAY. 

(4)  Stanford  Programmers  use  DOUBLE  on  occassion,  while 
Lockheed  Programmers  almost  never  use  it. 

(5)  Lockheed  Programmers  produce  about  2-1/2  times  as  many 
comments  as  Stanford  ones. 

(6)  The  large  0.  5 probability  of  Stanford's  assignment  statements 
seems  to  indicate  that  the  Stanford  code  is  more  simplistic  (for 
better  or  worse?  ) than  the  Lockheed  code. 

The  most  amazing  fact  is  that  with  all  these  differences,  the  data  is  rea- 
sonably well  fitted  by  a two  segment  piecewise  linear  Zipf  law  over  many 
decades,  and  the  constants  C and  B are  almost  identical  for  both  plots. 

Based  on  the  above  data  and  our  previously  drawn  analogies  between 
computer  and  natural  languages  it  is  fair  to  conclude  that  computer  pro- 
grams do  follow  Zipf  s law  and  its  extensions.  Conclusions  on  how  closely 
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the  fit  is  in  general,  and  whether  there  are  any  major  exceptions  or  classes 
of  data  which  fit  particular  forms  of  the  model,  must  await  further  study  of 
data.  The  use  to  which  we  put  our  Zipf  s law  models  is  discussed  in  the 
following  section. 

We  can  explore  how  well  the  formulas  for  the  constant  C and  the  num- 
ber of  tokens  n estimate  the  actual  values  observed  in  the  data.  In  Table 
13,  a comparison  is  made  for  4 cases  between  the  calculated  value  of  p^=  C. 
and  the  experimental  value  p,  =nj/n,  and  the  model  abcissa  intercept. 
Similarly  we  compare  the  actual  value  of  n with  the  calculated  estimates. 

In  all  cases  the  agreement  is  reasonable.  Additional  data  comparing  cal- 
culated and  actual  values  of  n is  given  in  Sec.  3.  4. 

Table  13 

Camparison  of  Calculated  vs.  Experimental 
Values  of  p^  and  n 


Fig.  No. 

t 

Calculated 

1 

0.  5772+lnt 

*^1 

Pi  = — 

Model 

Intercept 

Calculated 
t(0.  5772+lnt) 

Data 

n 

■7 

12 

0.  32 

0.  35 

0.  3 

37.  5 

31 

8 

9 

0.  36 

0.  25 

0.  4 

25 

24 

9 

21 

0.  28 

0.  20 

0.  35 

75 

55 

51 

0.  22 

0.  16 

0.  25 

23 

222 

3.  3 

Laws 

of  Complexity 

and  Length 

The  fact  that  Zipf's  law  can  be  applied  to  computer  programs  is  of  in- 
terest but  of  little  practical  importance  by  itself.  Also  being  able  to  cal- 
culate program  length  from  the  number  of  types  appearing  in  the  program 
seems  unimportant  unless  the  relation  used  forms  a part  of  a more  com- 
prehensive theory  of  program  complexity.*  Such  a theory  is  being  devel- 
oped,^ 9 and  a sketch  will  be  given  here.  It  is  hoped  that  by  this  means  the 
number  of  programming  errors,  development  effort, etc.  , can  be  related  to 
other  parameters. 

Briefly,  a computer  program  indicates  to  the  computer  which  function 
to  use  in  mapping  input  data  to  output  data.  The  fiinctions  of  practical  in- 
terest are  defined  in  terms  of  simpler  functions,  and  these  in  terms  of  yet 
simpler  functions,  etc.  , until  functions  are  reached  which  are  evaluated  by 

Of  course  there  are  many  obvious  uses  of  such  a length  measure  which  sug- 
gest themselves  even  without  such  a general  theory:  (1)  to  use  as  an  im- 
proved normalizing  factor  instead  of  statement  length  in  comparing  debug- 
ging data^l,  (2)  as  a managerial  tool  in  comparing  and  ranking 
programs  by  complexity,  (3)  as  a parameter  to  use  in  correlation  studies 
of  experimental  data  on  number  of  bugs,  man  hours,  etc. 
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the  computer  hardware  or  by  system  subroutines.  Tlie  complexity  of  a 
function  can  be  measured  by  the  extent  to  which  it  can  be  d<Tomposed  into 
simpler  functions,  as  is  well  known  in  switchinfj  theory  and  recursive  function 
theory.  The  number  of  levels  in  the  functional  decomposition  will  be  related 
to  the  number  of  variable  and  procedure  names  used  in  the  program.  Some 
interesting  preliminary  results  have  been  obtained  on  the  tradeoffs  between 
orogram  length,  number  of  input  and  output  bits,  hardware  complexity, 
storage  requirements,  and  execution  time.  19  por  example:  i)  the  product 
of  execution  time  and  hardware  complexity  is  approximately  constant; 

2)  a minimum  expected  program  length  can  be  defined,  but  it  depends  on  the 
probability  of  using  the  function,  not  its  complexity;  3)  trying  to  reach  the 
minimum  program  length  might  require  excessively  large  "compiler"  com- 
plexity and  storage  space.  Some  definite  bounds  relating  the  above  para- 
meters have  been  obtained  and  will  be  reported  elsewhere  (Ref.  19).  Of 
course,  the  quantitative  results  apply  strictly  only  to  the  stated  mathematical 
model  and  their  application  to  actual  programming  situations  will  have  to  be 
studied. 

One  method  of  initially  estimating  program  length=:=  (number  of  tokens) 
is  to  estimate  the  number  of  types.  We  assume  the  analyst  initially  has  a 
complete  description  of  the  problem  and  that  a partial  analysis  and  choice  of 
key  algorithms  has  been  made.  An  elementary  approach  might  be  to  esti- 
mate the  token  size  by 

(1)  Estimating  the  number  of  operator  types  which  will  be  used  in  the 
language  by  the  assigned  programmers. 

(2)  Estimate  the  number  of  input  variables,  output  variables,  inter- 
mediate variables,  and  constants  which  will  be  needed. 

(3)  Sum  the  estimates  of  step  (1)  and  (2)  and  substitute  in  Eqs.16,  20. 
26, or  34. 

3.4  Relationship  to  "Software  Physics" 

The  initial  motivation  for  the  application  of  Zipf  s law  to  computer 
languages  came  from  a review  of  Halstead' s^^  work  on  Software  Physics. 
Early  in  his  work  he  arrives  at  a formula  for  program  length 

L=  71^  log^  + Tl^log^  (39) 

where 

L = Program  length 
Tly  Number  of  operator  types 

71^  Number  of  operand  types 


In  addition  one  must  add  other  classes  of  statements  and  programming 
elements  such  as;  comments,  declares,  ce rtain  assemble r directives , 
etc. 


In  terms  ot  our  notation  the  analogous  quantities  are 


(40) 

(41) 


t=  T/j  + 71^ 

i\  = L. 


Note  that  Eq.  39  and  Eqs.  14  and  ZO  are  of  similar  form.  In  Table  14,  we 
compare  the  actual  number  of  tokens  with  the  number  of  tokens  calculated 
using  Eqs.  14,  ZO,  and  39.  Both  the  average  error  and  average  magnitude 
error  are  computed.  Both  Eqs.  14  and  39  yield  good  agreement  between 
actual  and  calculated  results. 


Conclusion 


A connection  has  been  established  between  some  well-known  results  in 
the  statistics  of  natural  languages  (Zipf's  law)  and  some  recent  observations 
on  the  relations  between  various  parameters  of  computer  programs,  mostly 
initiated  by  the  work  of  Halstead.  Also,  a mathematical  model  is  presented 
which  can  yield  both  the  original  Zipf  law  and  the  binomial  distribution  as 
special  cases.  This  derivation  is  a generalization  of  that  of  Mandelbrot.  The 
model  can  be  applied  to  such  diverse  processes  as  the  generation  of  a com- 
puter program  by  a contex  free  grammar,  and  to  the  summation  of  random 
variables  (central  limit  theorem).  Work  is  continuing  on  relating  the  length 
messages  obtained  to  the  number  of  programming  bugs  and  man-hours  of 
effort. 


39 


LawsoQ’ll 
Statements  PL/l 

55 

72 

+ 31 

76 

+38 

46 

-16 

Lawson -2T  1 

Statements  PLVl 

222 

240 

+8 

230 

+3.6 

140 

-37 

\tlKBUG-M6800 

Micro 

Exec.  Program 

87 

481 

+49 

378 

+17 

230 

-29 

Halstead  Ex. 

Ref.  (17),  p.  10 

52 

49 

-6 

! 

32 

-38 

20 

-62 

Halstead,  Ref.  j 
(17)  CACM#1 

104 

104 

0 

109 

♦5 

66  1 

-36 

CACM  #2 

82 

77 

-6 

89 

+ 8 

54 

-34 

CACM  #3 

453 

300 

-34 

275 

-39 

167 

-63 

CACM  #4 

132 

139 

+5 

155 

+ 17 

94 

-28 

CACM  #5 

123 

123 

0 

124 

+ 1 

75 

-39 

CACM  #6 

98 

101 

■r3 

+ 12 

61 

-37 

CACM  #7 

59 

62 

+ 5 

67 

+ 13 

40 

-31 

CACM  #8 

131 

-11 

124 

-5 

75 

-43 

CACM  #9 

314 

288 

-8 

280 

-11 

170 

-46 

CACM  #10 

46 

52 

+ 13 

62 

+ 35 

38 

-17 

CACM  #11 

S3 

52 

-1 

S>2 

+ 17 

38 

-28 

CACM  #12 

59 

62 

+5 

71 

+ 20 

43 

-27 

CACM  #13 

59 

57 

-3 

71 

+ 20 

43 

-17 

CACM  #14 

186 

163 

-12 

170 

-9 

103 

-45 

Average 

Error 

- 

- 

+ 2.  1% 

- 

+5.8% 

- 

-33% 

Average 

Magnitude 

Error 

- 

10% 

■ 

17.  2% 

1 

- 

33% 

Appendix  A - Derivation  of  Zipf  s and  Other  Related  Laws 


A.  1 Origins  of  Zipf's  Law 

By  Zipf  s law  we  mean  a rank-probability  distribution  such  as  p = c/r, 
or  a generalization  such  as  p = C/(A+r)®  , or  a segmented  straight  line 
function  on  log -log  paper.  This  law  is  usually  attributed  to  Zipf  in  con- 
nection with  word  frequencies;  however,  others  have  priority  not  only  in 
linguistics  but  also  in  other  areas  such  as  incomes,  particle  size,  popula- 
tion, etc.  An  analogy  may  be  drawn  to  the  central  limit  theorem,  in  that 
many  physical  processes  in  several  fields  imply  a simple  common  probabil- 
ity distribution.  The  advantages  in  using  such  a distribution,  if  it  is  shown 
to  be  valid  in  the  area  under  investigation,  is  that  such  properties  as  infor- 
mation content  (entropy)  or  the  type-token  ratio  may  be  calculated  from  2 
to  3 easily  estimated  parameters  instead  of  from  perhaps  thousands  of 
individual  probabilities.  A model  will  be  introduced  which  yields  both  Itipf  s 
law  and  the  binomial  distribution  as  marginal  cases.  The  model  is  a 
generalization  of  that  suggested  by  Mandelbrot  and  others. 

A.  2 Linear  Graph  Model 

A linear  graph  model  will  be  described  which  can  give  rise  to  any 
probability  distribution  over  a countable  number  of  events,  then  certain 
particular  cases  will  be  identified  with  some  familiar  probability  distri- 
butions. and  finally  several  physical  interpretations  of  the  linear  graph  will 
be  given.  Consider  a directed  linear  graph  with  no  circuits  (paths  returning 
to  their  starting  node)  and  with  but  one  origin  (node  with  only  leaving 
branches).  Divide  the  nodes  into  3 classes:  an  origin  node  which  has  only 
branches  leaving  it.  terminal  nodes  which  have  no  branches  leaving  them, 
and  intermediate  nodes  which  have  one  or  more  branches  entering  frorn 
other  nodes  and  leaving  for  other  nodes.  Each  intermediate  node  j will 
have  associated  with  it  a probability  distribution  Pj^{j)  giving  the  probability 
of  leaving  by  one  of  the  nj  branches: 


Zp^d)  = 1 


(A.  1) 


Thus  each  path  from  the  origin  passing  thru  nodes  1 (the  origin). j2  , . . . , 

has  a probability 


Pi  (1)  Pi  (j?)  • • • P:  n ) 

h ^2  ^ ^m-r^m-r 


(A -2) 


provided  that  the  branch  decision  probabilities  at  each  node  are  independent. 
Finally,  each  node  will  have  associated  with  it  a probability  p • which  is  the 
sum  of  all  of  the  path  probabilities  from  the  origin  to  it.  The  Sum  of  the  pj 
over  the  terminal  nodes  (index  set  T)  is  clearly  unity: 
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(A.  J) 


\' 

jc“T 


It  is  the  distribution 


"j 


which  will  be  of  interest  here. 


It  can  easily  be  shown  that  the  branch  proba\)ilitics  p.(  j)can  be  chosen  s«> 
as  to  yield  any  desired  probability  distribution  Pj.  This  will  usually  re- 
quire the  Pj^{j)  to  be  different  for  each  node  j.  If  there  are  only  a few  dif- 
ferent distributions  Pi(j)  allowed,  then  the  p j over  the  terminal  nodes  will 
belong  to  one  of  a few  regular  distributions. 


A.  3 Generating  Graph  is  a Tree 

If  the  generating  graph  has  no  reentrant  paths,  i.  e.  , if  there  is  a unique 
path  from  the  origin  to  a given  node,  then  the  resulting  probability  distribu- 
tion p-  is  easier  to  characterize,  and  it  will  be  a Zipf  type  if  the  resulting 
tree  is'^homogeneous  (branch  probabilities  independent  of  node). 

The  probabilities  Pj  are  easy  to  calculate  in  rank  order  in  this  case 
i.  e.  , so  that 


p ^ > P^>  P3  > • • . . (A.  4) 

This  calculation  is  facilitated  by  associating  with  each  branch  a length 

fn  p = i.(j).  Then  the  log  probability  associated  with  a node  is  simply 

the  (unique)  distance  from  the  origin.  If  the  tree  is  homogeneous,  let  n be 
the  number  of  branches  leaving  each  intermediate  node  and  going  to  another 
intermediate  node,  and  let  be  their  lengths  with 

12  n 

The  number  of  intermediate  nodes  at  a distance  of  x or  less  from  the 
origin,  R(x),  will  then  satisfy  the  following  difference  equation 

n 

R(x)  = J]  R(x  - f .)  + 1 X > 0 

1=1 

R(x)  =0  X < 0 


(A.  5) 


This  can  be  proved  by  noting  that  every  intermediate  node  (except  the  origin) 
within  radius  x is  contributed  by  a branch  of  some  length  fj  and  by  no 
other.  The  above  difference  equation  has  a solution  which  is  asymptotic  to 
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(A.  6) 


R (x)  = e 

0 /. 

i 

P,  “ g 

where  B is  given  by  ^ e = I 

i=l 


and  in  fact  (x)  is  a good  smoothed  approximation  to  the  discontinuous 
step  function  R(x)  for  all  x. 

Now  let  there  be  t branches  going  to  terminal  nodes  from  each  inter- 
mediate node  (including  the  origin),  and  let  their  probabilities  be 


^n+2 Pn-l-t 


We  must  have 


n n+t 

E Pi  + x Pi 

i = l ^ i=n-(-l 


n - f . n+t  - 1 . 

i=l  i=n+l 


(A.  7) 


(A.  8) 


and  therefore  B is  greater  than  1 if  there  are  to  be  non-zero  terminal  prob- 
abilities. The  total  number  of  terminal  nodes  at  distance  x or  less  from 
the  origin  is  now 


i=n+l 


R(x  - fj) 


Using  the  smoothed  approximation  to  R,  and  noting  that  x = T — 

desired  rank  probability  relationship  is  approximately  ^r 


(A.  9) 


. the 


i=n+l 


p = 

B 

r 


where  C 


i=n+l 


(A.  10) 


A better 
where  all  p. 


feeling!  for  the  value 
are  e<^ial  (to  ^ ■ ) 


B may  be  obtained  in  th»-  special  i.»>« 
. In  this  case 


/ntn-t-t) 
In  n 


(A.  11) 


A.  4 Generatinji’  Graph  for  the  Binomial  Distribution 

As  a contrast  to  the  tree  just  discussed,  consider  a generatin)*  ):raph 
such  as  shown  in  Fig.  18.  Here  all  terminal  nodes  are  at  the  lowest  level, 
and  there  are  many  reentrant  paths.  It  can  be  seen  that  the  number  of  paths 
to  the  i^^  node  at  the  n^^  level  is  (H)  . and  that  if  the  downward  choices 
are  made  with  equal  probability,  the  terminal  probabilities  are 
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2 


n 


(A.  12) 


This  is  maximum  for  i=n/ 2,  and  it  decreases  towards  the  side  approximately 
according  to  a Gaussian  distribution 


(A.  1 i) 


• B 

This  decreases  very  rapidly  compared  to  r . The  reentrant  paths  cause 
the  rapid  decrease  because  they  make  it  less  probable  that  a given  traversal 
will  reach  the  outermost  nodes. 

The  completely  reentrant  graph  of  the  type  shown  in  Fig.  18  can  be 
described  in  terms  of  either  a random  walk  or  a summation  of  random 
variables  (sideways  unit  deflections).  Thus,  this  case  illustrates  the  cen- 
tral limit  theorem. 


A.  5 Physical  Interpretation  of  Generating  Graph 

The  original  interpretation  of  the  tree  by  Mandelbrot  was  the  generation 
of  a word  of  a natural  language  by  adding  a letter  (n  = 26)  or  a word-terminat- 
ing space  (t=l).  The  reentrant  graph  of  Fig.  19  could  represent  the  random 
walk  followed  by  a ball  falling  through  Galton' s pegboard  (an  educational  aid 
for  demonstrating  the  Gaussian  distribution).  Returning  to  the  tree,  it 
might  represent  the  repeated  division  of  a whole  into  parts,  thus  explaining 
Zipf-like  distributions  of  income,  sand  particle  sizes,  etc. 

More  to  the  point  here,  the  tree  might  represent  a repeat  of  the  pro- 
duction rules  of  a context  free  grammar.  A simple  example  is  the  palin- 
drome generator  (Fig.  18): 
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(\-2  t = 3 


FIG.  18.  INFINITE  HOMOGENEOUS  TREE 
GIVING  A ZIPF  DISTRIBUTION. 


FIG.  19.  FINITE  REENTRANT  GRAPH 
GIVING  THE  BINOMIAL 
DISTRIBUTION. 


FIG.  20.  SPECIAL  CASE  OF  TREE  WHICH  FIG.  21.  FINITE  TREE  GIVEN  THE 

^ GIVES  GEOMETRIC  DISTRIBU-  UNIFORM  DISTRIBUTION. 

TION. 


FIG.  22.  REENTRANT  PATHS  IN  GENERATING 
AN  ALGEBRAIC  EXPRESSION. 
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a 0 a 0 


a 1 

a 

Lf  the  probability  of  iisin^  each  rule  is  equal  (to  l/5)  for  all  steps,  then  B 
is  tv.S / InZ  = Z.  32  by  solving 


(A.  14) 

= 3 


2 e 


/n5 

B 


1 


(A.  15) 


and  the  approximate  distribution  is 


P 

r 


_C 

2.  32 


(A.  16) 


Probably  the  most  succesful  theory  of  natural  languages,  and  certainly 
of  computer  languages,  from  the  mathematical  point  of  view  is  that  started 
by  Chomsky.  The  Backus  Normal  Form  has  become  the  standard  way  to 
formalize  the  grammar  of  computer  languages.  Consider  the  fragment 

(expression>  : = expression  { + |-|/|*}  <,expression> 

<pxpression>  ; =A1b|c1d 

Let  e represent  ^expressions  and  consider  the  derivations 

e =e^e  - e=i>e  - C=J>e  + e - C =i>A  + e - C =i>A  + B - C 
e =i>e  - e =i>e  + e - e =?-A  + e - e =i>A  + B - e =e>A  + B - C 

This  illustrates  that  reentrant  path  corresponds  to  different  ways  to  derive 
the  same  expression.  In  a frequency  count  there  is  no  way  to  distinguish  the 
A+B-C's  which  were  generated  in  these  two  ways,  and  usually  in  many 
others.  This  example  does  not  illustrate  an  ambiguous  grammar,  since 
only  the  second  derivation  is  a left  most  derivation.  Since  reentrant  paths 
can  occur  even  if  the  language  is  inherently  unambiguous,  the  cases  between 
the  tree  and  the  Gallon  board  are  of  paramount  interest  in  studying  the 
statistics  of  the  elements  of  a programming  language. 

A.  6 Remark  on  "Least  Effort" 


Several  authors  have  explained  Zipf  s law  as  a result  of  a information- 
maximizing  or  cost -minimizing  process,  but  this  seems  to  either  involve 
circular  reason,  or  to  beg  the  question.  The  argument  may  be  simplified 
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to  something  like  this:  let  c.  be  the  cost  associated  with  the  i'-"  word, 
perhaps  representing  its  length.  If  the  c^  are  subject  to  constraints  such 
as  requiring  that  no  word  begin  another  word.  Shannon  has  shown  that 


H = log  ^ p.  c.  = C 

1 * 1 I 


(A.  17) 


If  we  wish  to  minimize  C with  fixed  H the  optimum  value  of  c.  is 


c . = log  

1 ^ p. 


(A.  18) 


The  same  relation  should  hold  if  we  wish  to  maximize  H with  fixed  C,  or 
to  maximize  H/C.  ^ somehow  it  is  known  that 


Cj  = log  k i 


(A.  19) 


then  p.  follows  Zipf  s law.  This  merely  transfers  the  problem  to  ex- 
plaining why  C£  varies  logarithmically,  apd  in  fact  this  is  usually  done  by 
a generating  tree  such  as  described  above. 
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of 
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RAVC  plans  and  conducts  research,  exploratory  and  advanced 
development  programs  in  command,  control,  and  conounicatlons 
(C^ ) activities , and  in  the  areas  of  informatior,  sciences 
and  intelligence.  The  principal  technical  mission  areas 
are  contmmications , electromagnetic  guidance  and  control, 
surveillance  of  ground  and  aerospace  objects,  intelligence 
data  collection  and  handling,  inforxtation  system  technology, 
ionospheric  propagation,  solid  state  sciences,  microeave 
physics  and  electronic  reliability,  maintainability  and 
compatibility . 


